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ABSTRACT 

D. B. Bailey and S. A. Palsha (1992) proposed two 
modified versions of the Stages of Concern Questionnaire for 
measuring teacher concerns during a reform effort. Their analysis 
suggested the use of a 5-factor model with 35 items or 15 items 
rather than the original 7-stage, 35-item Concerns Based Adoption 
Model (CBAM) . The present study was carried out in two phases. The 
first phase analyzed data from 376 algebra teachers to evaluate 
reliability and validity issues of both the original Stages of 
Concern Questionnaire and the two proposed versions of Bailey and 
Pasha. Reliability estimates were lower for the 15-item revised 
version, but agreed with Bailey and Palsha for the original 
instrument and the 35-item, 5-factor version. A confirmatory factor 
analysis was conducted using the 5-factor, 35-item version. A less 
than optimal fit of the model to the data indicated continued 
problems with validity. From factor loadings and modification 
indices, a 27-item 5-stage model was proposed that more closely 
matched the original CBAM without the Awareness and Refocusing 
Stages. The second phase of the study employed a confirmatory factor 
analysis of the 27~item 5-stage model to examine data from a new 
sample of 237 algebra teachers. Reliability estimates were 
consistently higher than for other versions, but validity continued 
to be problematic. A broad range of issues arising from these results 
is discussed. (Contains 8 tables and 14 references.) (Author/SLD) 
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Abstract 

Bailey and Palsha (1992) proposed two modified versions of the Stages of Concern 
Questionnaire for measuring teacher concerns during a reform effort. Their analysis 
suggested the use of a five factor model with 35 items or 15 items rather than the original 
seven stage, 35-item Concerns Based Adoption Model. The present study was carried out 
in two phases. The first phase analyzed data from 376 algebra teachers to evaluate 
reliability and validity issues of both the original Stages of Concern Questionnaire and 
the two proposed versions of Bailey and Palsha. Reliability estimates were lower for the 
15-item revised version, but agreed with Bailey and Palsha for the original instrument 
and the 35-item, five factor version. A confirmatory factor analysis was conducted using 
the five factor, 35-item version. A less than optimal fit of the model to the data indicated 
continued problems with validity. From factor loadings and modification indices, a 27- 
item five stage model was proposed that more closely matched the original CBAM 
without the Awareness and Refocusing Stages. The second phase of the study employed 
a confirmatory factor analysis of the 27-item five stage model to examine data from a 
new sample of 273 algebra teachers. Reliability estimates were consistently higher than 
for other versions, but validity continued to be problematic. A broad range of issues 
arising from these results are discussed. 
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Background 

Current literature on teacher change emphasizes that change is a process not a 
single event (Friel, 1993; Fullan, 1991; Guskey, 1986; Hall & Hord, 1987). The 
Concerns Based Adoption Model (CBAM) provides a developmental framework for the 
role of teacher concerns in this change process. Designed in the early 1970's, the Stages 
of Concern Questionnaire (SoCQ) assesses the level of concern related to implementing 
an educational innovation (Hall, George and Rutherford, 1986). With the influx of 
current curriculum reform efforts, the questionnaire has been increasingly utilized in staff 
development projects as either a diagnostic or evaluative instrument for facilitating 
change (ERIC search). A recent book, Staff development for education in the ‘90s 
features the CBAM in a lead chapter (Loucks-Horsley & Stiegelbauer, 1991), describing 
its use in a wide variety of professional development environments. 

As the SoCQ is used in an ever widening array of education and training settings 
it is vital to consider the extent to which results from the questionnaire are providing 
inservice coordinators and leaders accurate information about teacher concerns over 
proposed innovations. A study by Bailey and Palsha (1992) raised questions about both 
the reliability and validity of the seven scale model as a measure for teacher concerns. 
Using the same 35-item Stages of Concern Questionnaire (SoCQ), they suggested a 
revised five factor model. In addition, Bailey and Palsha hypothesized a shortened 15- 
item five factor questionnaire. The present study was carried out in two phases. The 
first phase analyzed data gathered from 376 algebra teachers who had taken the 35-item 
SoCQ as a part of a seven day inservice training program conducted during summer, 
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1992. The authors conducted reliability and validity studies in order to independently 
verify the work of Bailey and Palsha on their proposed revised questionnaires. From 
these analyses, a 27-item five stage model was suggested that might enhance the 
reliability and validity of the SoCQ. In the second phase, data were analyzed from a new 
sample of 273 algebra teachers involved in the same seven day inservice training 
conducted during summer, 1993. Reliability and validity analyses were carried out to 
attempt to verify the results from the first phase of the study. 

Theoretical Framework 

Based upon work by Fuller (1969), the Concerns Based Adoption Model proposes 
seven stages of concern as teachers implement a new innovation: Awareness, 

Information, Personal, Management, Consequence, Collaboration, and Refocusing. 
Research conducted as a part of the initial instrument development utilized principal 
component factor analysis with varimax rotation to reduce 150 original items from 363 
respondents into the seven subscales. Validity and reliability issues addressed in a two 
and a half year study are reported by Hall, George, and Rutherford (1986). This work 
was followed by studies to verify the existence of the stages as a developmental process 
(Hall & Loucks, 1978). Since its development, the SoCQ has been used in more than 
fifty studies across disciplines to measure the level of teacher concerns with 
implementing an innovation (ERIC search). 

A recent study by Bailey and Palsha (1992) conducted a factor analysis with data 
collected from 142 professionals working in intervention programs serving children with 
disabilities and their families. Their findings did not support the original 35-item, seven- 
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stage CBAM model. The authors found low reliability estimates for the Awareness stage 
(.42) and the Refocusing stage (.61). Five factors were found to provide the best model 
to fit the data for the 35 items. The authors identified the five reorganized stages as 
Awareness, Personal, Management, Impact, and Collaboration. In addition, the authors 
proposed a second, shorter 15-item SoCQ. Elimination of items was based upon low 
factor loadings, theoretical consistency with a factor, and equalization of the number of 
items per factor. A correlation between the five scale 35-item form and the five scale 15- 
item form revealed high compatibility for each factor (.87 to 1 .00). The assignment of 
the 35-items from the SoCQ to the various stages proposed in each model is outlined in 
Table 1. 

The Bailey and Palsha factor analysis produced a five factor, 35-item model with 
reliability estimates of .83, .81, .79, .74, and .60. Their reduced 15-item instrument 
contained reliability levels of .76, .73, .78, .74 and .55. Reliability estimates were 
somewhat lower for the five scale 15-item SoCQ but the authors considered these not to 
be “appreciably” lower. To check the validity of the five-factor 15-item revision, an 
exploratory factor analysis was applied to a random subset of 75 subjects from the 
original sample. Results showed the five factors did not change and no significant 
multiple loadings were found. The researchers suggested further studies were needed to 
generalize these results across innovations and trainees. 
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Table 1 Assignment of Items to Proposed Stages of Concern for the 
Original CBAM and the Modified-CBAM 



Item 


CBAM 


Modified CBAM 


1. Don’t know what the innovation is * 


Awareness 


Awareness 


2. Not concerned about the innovation 


Awareness 


Management 


3. Occupied with other things 


Awareness 


Management 


4. Concerned with other things * 


Awareness 


Awareness 


5. Not interested in learning 


Awareness 


Management 


6. Limited knowledge about innovation * 


Information 


Awareness 


7. Like to discuss possible use 


Information 


Personal 


8. What resources are available 


Information 


Personal 


9. Immediate requirements for use * 


Information 


Personal 


10. How better than current practice 


Information 


Personal 


1 1 . Effects on professional status 


Personal 


Personal 


12. Who will make the decisions 


Personal 


Personal 


13. How my tasks will change * 


Personal 


Personal 


14. Time and energy required 


Personal 


Personal 


15. How my role will change * 


Personal 


Personal 


16. Not enough time * 


Management 


Management 


17. Conflict between interests and 
responsibilities * 


Management 


Management 


18. Inability to manage 


Management 


Personal 


19. Time spent on non-essential tasks 


Management 


Impact 


20. Coordination of tasks and people * 


Management 


Management 


21. Students’ attitudes toward innovation 


Consequences 


Impact 


22. How it affects students 


Consequences 


Impact 


23. Evaluating impact on students * 


Consequences 


Impact 


24. Exciting students about their part * 


Consequences 


Impact 


25. Use of feedback to change * 


Consequences 


Impact 


26. Help others with innovation * 


Collaboration 


Collaboration 


27. Develop relationships with others * 


Collaboration 


Collaboration 


28. Share progress with others * 


Collaboration 


Collaboration 


29. Coordinate efforts with others 


Collaboration 


Collaboration 


30. Know about others’ work 


Collaboration 


Impact 


3 1 . Other approaches that are better 


Refocusing 


Impact 


32. Revising use 


Refocusing 


Management 


33. Revising the approach 


Refocusing 


Impact 


34. Modify based on use 


Refocusing 


Impact 


35. How to change 


Refocusins 


Personal 



* Those items retained for the shorter 15-item version (stages stay the same) 
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Method 

The subjects for the first phase of the present study were 376 algebra teachers who 
participated in a seven day inservice program at one of sixteen sites across North Carolina 
during summer, 1992. The subjects for the second phase of the study were 273 algebra 
teachers who participated in the same seven day inservice program at one of seventeen 
sites across North Carolina during summer, 1993. These workshops were a joint effort by 
the North Carolina Department of Public Instruction and the Center for Mathematics and 
Science Education at the University of North Carolina at Chapel Hill, intended to retrain 
mathematics teachers in revised pedagogy and curriculum for first year algebra. Both 
samples could be described as secondary and middle grades teachers with a primary 
interest in algebra and varying backgrounds regarding the new curriculum topics. 

Teachers completed the 35-item SoCQ form on the first day of the training. The 
questionnaire contained Likert items with seven response categories ranging from 0 = 
irrelevant to me, 1 = not true of me now, to 7 = very true of me now. Written examples 
demonstrating how to classify degrees of concern were given to the teachers preceding 
the administration of the questionnaire. 

The first phase of the study focused on generating reliability and validity data for 
the original SoCQ and the two revisions of the SoCQ suggested by Bailey and Palsha 
(1992). Using SPSS-X Release 4.1 for VAX/VMS (SPSS, 1990), Cronbach-alpha 
coefficients were computed as measures of internal consistency for the seven factors 
hypothesized by the original CBAM and the reduced five factor solutions proposed by 
Bailey and Palsha. Based on low reliability estimates for some factors of both the 
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original CBAM and the 15-item revision, a confirmatory factor analysis was only carried 
out on the 35-item, five factor SoCQ suggested by Bailey and Palsha to obtain goodness- 
of-fit indices. The analysis employed the VMS Prelis and LISREL 7 statistical packages 
(Joreskog & Sorbom, 1990) ). 

Results of the confirmatory analysis were also inspected to determine whether 
particular items among the 35 questions might either be eliminated from consideration or 
reassigned to different factors, thereby improving the validity of the overall instrument. 
Based on information obtained from both the standardized solutions and the modification 
indices generated by the analysis, a revised 27-item SoCQ emerged which would become 
the primary object of study of the second phase. 

The second phase of the study focused on the analysis of the 27-item SoCQ with 
an independent sample. First, Cronbach-alpha coefficients were calculated using SPSS- 
X Release 4.1 for VAX/VMS (SPSS, 1990) to investigate internal consistency of the 
factors hypothesized by the original CBAM, the two five factor versions proposed by 
Bailey and Palsha (1992) and the 27-item SoCQ proposed by the present authors. 

Second, VMS Prelis and LISREL 7 (Joreskog & Sorbom, 1990) were utilized to conduct 
a confirmatory factor analysis investigating the validity of the 27-item, five factor model 
for measuring teacher concerns. 

Results and Conclusions 

First Phase Reliability Study 

Reliability estimates for the three versions of the CBAM are displayed in Table 2. 
For the original CBAM, reliabilities of the Awareness and Refocusing subscales were 
very low (.45 and .52, respectively). These findings agreed with the results of Bailey and 
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Palsha (1992) who found reliabilities for Awareness and Refocusing stages to be .42 and 
.61, respectively. For the modified five-factor CBAM, general improvement in the 
Cronbach alpha values was found over those of the original model (Table 2). This 
outcome agrees with the results of Bailey and Palsha. 

When reliability estimates were calculated for the same five subscales using 
Bailey and Palsha's 15-item shorter version of the SoCQ, the Cronbach-alpha values were 
lowered for four of the five stages. Two subscales had particularly low values, .57 for 
management and .50 for impact. Given these lower reliabilities for the proposed 15-item 
questionnaire as well as for the two subscales of the original CBAM, the present authors 
decided to carry out a validation study only for the 35-item, five factor modified-CBAM. 

Table 2 Cronbach-alpha Values for the Original 35-item CBAM, 
the 35-item Modified-CBAM and 15-item Modified CBAM 



CBAM Stages 


Modified-CBAM Stages 


35-items 


15-items 


Awareness .45 


Awareness 


.71 


.71 


Information .66 


Personal 


.82 


.79 


Personal .72 


Management 


.63 


.57 


Management .69 


Impact 


.68 


.50 


Consequences .60 


Collaboration 


.76 


.71 


Collaboration .77 








Refocusing .52 









First Phase Confirmatory Factor Analysis 

The 35-item, five stage model was submitted to a confirmatory factor analysis to 
determine the extent to which the items of the questionnaire are appropriate measures of 
the hypothesized stages. Skewness and kurtosis information for each of the 35 variables 
indicated that the data did not violate normality assumptions of the analysis. Prelis 
produced a variance-covariance matrix, with an effective sample size of N=376, which 
was used as the input matrix for a maximum likelihood extraction procedure in LISREL. 
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One variable for each of the five factors was set as a reference indicator, and its loading 
was fixed to be 1.0. Values for the remaining 30 factor loadings and their error variances 
were estimated as a part of the analysis (Long, 1983). 

LISREL produces four goodness-of-fit measures: Chi-square, Goodness of Fit 
Index (GFI), Adjusted GFI (AGFI), and Root Mean Square Residual (RMSR). The 
values for these indices are displayed in Table 4. Since the RMSR is difficult to interpret 
when using a variance-covariance input matrix as used in this confirmatory factor 
analysis, it will not be considered here. With the Chi-square measure, a value close to 
the degrees of freedom of the model would indicate a good fit to the data. In this case, 
the Chi-square value is much larger. However, a Chi-square measure tends to be inflated 
by large sample sizes and should be considered in conjunction with the GFI and AGFI 
measures. 

The GFI represents the ratio of squared residuals to observed variances and can 
take on values between zero (poorest fit) and one (perfect fit). The AGFI is a type of 
parsimonious fit index, which adjusts GFI for degrees of freedom. Typically, AGFI will 
be comparable with GFI unless an unusually large number of measures have been used in 
the model. In the case of an excellent fit of the model to the data, GFI would be at least 
0.90 and AGFI would be at least .80 (Miller & Thayer, 1989). Thus, the GFI and AGFI 
values in the table would be considered to be less than optimal. The smaller AGFI in this 
case indicates that, as Bailey and Palsha have theorized, an SoCQ with fewer items might 
be a more parsimonious measure of teacher concerns. 

Table 3 Goodness of Fit Measures 

Chi-square with 565 d.f. = 2307.24 (p=.000) 

GFI = 0.733 

AGFI = 0.702 

RMSR = 0,527 




11 



Concerns Based Adoption Model 



11 



Given the less than optimal fit of the model to the data, it was considered 
important by the present authors to investigate possible sources for the lack of fit. This 
type of inquiry is made feasible using the LISREL option for specifying modification 
indices for the model. These modification indices represent the amount the Chi-square 
index for the model would be decreased by allowing an item to be assigned to a different 
subscale. Table 4 displays the questionnaire items with modification indices greater than 



20 for a given factor. 

Table 4 SoCQ Items with Modification Indices Greater Than 20 for a Factor 



Item 


Aware 


Personal 


Manage 


Impact 


Collaborate 


Concerned about the area 


0.000 


20.478 


3.238 


9.671 


1.115 


Like to discuss possible use 


0.790 


0.000 


0.780 


15.368 


54.252 


Effects on professional status 


0.000 


0.000 


28.374 


0.720 


0.247 


Inability to manage 


0.005 


0.000 


49.572 


0.238 


10.971 


Time spent in nonessential tasks 


4.796 


24.274 


43.189 


0.000 


0.380 


Evaluating impact on students 


10.832 


20.462 


0.710 


0.000 


0.098 


Exciting students about their part 


0.391 


0.079 


11.647 


0.000 


26.122 


Share progress with others 


0.029 


4.512 


0.020 


24.758 


0.000 


Coordinate efforts with others 


1.091 


19.451 


0.703 


24.254 


0.614 


Know about others' work 


1.744 


27.860 


0.070 


0.000 


20.631 


Other approaches that are better 


96.383 


16.549 


0.086 


0.000 


8.407 


Revising use 


7.614 


40.022 


0.296 


22.792 


3.398 


Revising the approach 


31.876 


1.152 


2.069 


0.000 


6.513 


How to change 


3.074 


15.485 


5.153 


13.155 


22.279 



In considering the elimination or reassignment of measures of the various stages, 
it is important also to consider the loading of each item on its hypothesized factor. 

Table 5 provides the completely standardized solution for each item of the questionnaire. 
Smaller values indicate that the item might either be reassigned or eliminated, depending 
on the modification indices for the item. 
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Table 5 Standardized Solutions 



Item Aware 

Don't know what the innovation is .80 

Concerned with other things .57 

Limited knowledge about .64 



innovation 

Like to discuss possible use 
What resources are available 
Immediate requirements for use 
How better than current practice 
Effects on professional status 
Who will make the decisions 
How my tasks will change 
Time and energy required 
How my role will change 
Inability to manage 
How to change 
Not concerned about the 
innovation 

Occupied with other things 
Not interested in learning 
Not enough time 
Conflict between interests and 
responsibilities 

Coordination of tasks and people 
Revising use 

Time spent on non-essential tasks 
Students' attitudes toward 
innovation 

How it affects students 
Evaluating impact on students 
Exciting students about their part 
Use of feedback to change 
Know about others' work 
Other approaches that are better 
Revising the approach 
Modify based on use 
Help others with innovation 
Develop relationships with others 
Share progress with others 
Coordinate efforts with others 



Personal Manage Impact Collaborate 



.50 

.53 

.79 

.56 

.40 

.51 

.72 

.71 

.75 

.53 

.51 

.10 

.43 

.32 

.52 

.62 

.54 

.54 

.34 

.42 

.39 

.53 

.46 

.51 

.49 

.15 

.52 

.55 

.67 

.64 

.69 

.63 



If items are eliminated based upon a low standardized solution value for the 
assigned stage, a value < .50, and low values for modification indices, values < 20, five 
questions would be removed from the questionnaire. These include: 
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-Not concerned about the innovation (Management) 

-Occupied with other things (Management) 

-Not interested in learning (Management) 

-Students' attitudes toward innovation (Impact) 

-How it affects students (Impact) 

All three of the questions that are listed in the Management subscale above were 
questions that shifted from the Awareness stage to the Management stage in Bailey and 
Palsha's five factor model. It seems reasonable from the data to drop the three questions. 
These items were also eliminated by Bailey and Palsha when constructing their 15-item 
version of the questionnaire. Although the data suggest that the two questions from 
Impact be eliminated, the present authors feel that focusing on student learning is an 
important aspect of the CBAM concept and should remain in the model. 

Four other items have standardized solution values between .50 and .60 and 
modification indices less than 22. 

-What resources are available (Personal) 

—How better than current practice (Personal) 

-Modify based on use (Impact) 

-How to change (Personal) 

Due to the length of the Personal subscale, it was decided to eliminate the first two 
questions . The last two questions are items that shifted from the Refocusing factor to the 
Impact and Personal factors. Conceptually, these items are not consistent with the new 
factors and may be eliminated. All four of the above questions were also dropped by 
Bailey and Palsha for their 15-item questionnaire. By removing a total of seven 
questions, the new 5-factor modified CBAM would consist of 28 questions. 

The other consideration in modifying the model is the reassignment of items. If 
items are reassigned based upon low standardized solution values for the assigned stage, 
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< .55, but higher modification indices, > 20, six items would be reassigned on the 
questionnaire. By reassigning these items, the Chi-square index would decrease by the 
amount indicated by the modification index for the new factor. The original stage and 
suggested movement stage are indicated in the parenthesis, followed by the modification 
index for the new stage: 

— Like to discuss possible use (Personal— Collaboration, 54.3) * 

- Effects on professional status (Personal— >Management, 28.4) 

-Inability to manage (Personal— >Management, 49.6)* 

- Time spent on non-essential tasks (Impact— »Management, 43.2) * 

- Exciting students about their part (Impact— ^Collaboration, 26.1) 

- Know about others’ work (Impact— >Personal, 27.9) 

- Other approaches that are better (Impact— > Awareness, 96.4) * 

-Revising the approach (Impact— » Awareness, 31.9) 

Four of the above questions, “effects on professional status,” “exciting students about 
their part,” “knowing about others' work,” and “revising the approach” were considered 
by the authors to be conceptually linked to the hypothesized factors and therefore were 
not reassigned. These questions would not have as large an effect on the Chi-square 
index since they represented the smallest modification indices among the items. The 
starred questions above are items the authors decided to reassign. These movements 
better equalized the number of questions per scale. Reliability estimates were 
recalculated for the subscales. With the additional question, "Other approaches that are 
better," the Awareness subscale reliability lowered substantially (.11). If the item is 
moved back to Impact, the reliability for Impact is low (.59). The authors decided to drop 
this question, leaving the original three questions in Awareness and a total of 27 
questions for the new instrument. The final reliability estimates for the instrument were 
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Awareness=.71, Personal=.75, Management=.73, Impact=.66 and Collaboration=.77. 
The questions for each subscale are shown in Table 6. 

Table 6 Twenty-Seven Item Five Factor SoCQ 



Stage 1 - Awareness 

Don't Know What innovation is 
Concerned with other things 
Limited Knowledge about innovation 
Stage 2 - Personal 

Immediate requirements for use 
Effects of professional status 
Who will make the decisions 
How my tasks will change 
How my role will change 
Time and energy required 
Stage 3 - Management 
Not enough time 

Conflict between interests and responsibilities 
Inability to manage 
Coordination of tasks and people 
Time spent on non-essential tasks 
Revising use 
Stage 4 - Impact 

Students attitude toward innovation 
How it affects students 
Evaluating impact on students 
Exciting students about their part 
Use of feedback to change 
Revising the approach 
Know about others’ work 
Stage 5 - Collaboration 

Like to discuss possible use 
Help others with innovation 
Develop relationships with others 
Share progress with others 

Coordinate efforts with others 



Reliability estimates for the data from this sample did not support the 15-item 
five-factor CBAM proposed by Bailey and Palsha (1992). Although the idea of a much 
shorter 15-item SoCQ would be appealing for teachers completing the form and for 
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training coordinators analyzing SoCQ items, reducing the number of questions from 35 
to 15 with only three items per scale is tenuous, losing much of the detail of the 
conceptual model. The 27-item five-factor instrument proposed in this study eliminates 
unnecessary and inappropriate questions for the conceptual model without distorting the 
meaning of the subscales. The reliability estimates were sufficiently strong to warrant the 
use of the proposed 27-item questionnaire. 

This new 27-item SoCQ matches more closely the original CBAM, but without 
the Awareness and Refocusing stages that had low reliabilities. Three items from the 
original Awareness subscale that shifted to the Management scale were eliminated from 
Management in this study, as suggested by the analysis. Results of the analysis also 
suggested removal of three items from the original Refocusing subscale. The remaining 
two Refocusing items were now in the Management and Impact stages. The second 
phase of the study examined the reliability and validity of the new 27-item model. 

Second Phase Reliability Study 

Reliabilities for each of the models were computed using data from the second 
workshop. Reliability estimates are displayed in Table 7. With the new sample, 
reliabilities for the 27-item SoCQ remained stable. Several factors for the Bailey and 
Pasha models continue to be below .60. 

Second Phase Validity Study 

Prelis produced a variance-covariance matrix, with an effective sample size of 
N=241, which was used as the input matrix for a maximum likelihood extraction 
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procedure in LISREL. One variable for each of the five factors was set as a reference 

indicator, and its loading was fixed to be 1.0. Values for the remaining 22 factor loadings 

Table 7 Cronbach-alpha Values for the Original CBAM, the Different 
Versions of the Modified CBAM (27-item, 35-item, 15-item) 



Original CBAM 


27-Item, Modified 


35-Item, Modified 


15-Item, Modified 


Awareness = .42 


Awareness =.71 


.70 


.73 


Information = .58 


Personal = .76 


.80 


.68 


Personal =.71 


Management = .69 


.54 


.56 


Management= .63 


Impact = .68 


.71 


.51 


Consequence= .64 


Collaboration = .72 


.73 


.69 


Collaboration=.74 








Refocusins =.48 









and their error variances were estimated as a part of the analysis. Goodness-of-fit indices 
produced by the confirmatory factor analysis are shown in Table 8. Although the p-value 
suggests to reject the model, with a large degree of freedom the Chi-square value tends to 
be inflated. The Chi-square should be considered along with the GFI and ADFI. An 
excellent fit would have a GFI of .90 or above with a ADFI at least .80. The values for 
this model indicate a somewhat better fit than the confirmatory results for the 35-item 
five factor Bailey and Palsha model in the first phase study. However, the 27-item model 
remains a less than optimal fit. 

Table 8 Goodness of Fit Measures 



Chi-square with 329 d.f. 

GFI 

AGFI 

RMSR 



= 1043.85 (p=. 000) 
= 0.760 
= 0.724 

= 0.523 



An exploratory examination of the standardized solution values and modification 
indices suggested values for only two questions that might warrant movement or 
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omission of questions. The item, "I am concerned about time spent working with 
nonacademic problems related to the innovation" (Management stage), had a standardized 
solution value < .50 and modification indices < 20 indicating the question could be 
dropped. The item, "I would like to discuss the possibility of using the innovation" 
(Collaboration stage) had a standardized solution value < .50 with one modification index 
> 20 suggesting that the question could be moved to the Personal stage. However, these 
two changes would make very little difference in the Chi-square, GFI and AGFI results. 

In a similarly exploratory use of the LISREL procedure, the same 1993 data were 
analyzed using the original 35-item, seven scale CBAM. The goodness-of-fit values were 
consistently poorer than for the 27-item questionnaire: Chi-square = 1779.62 (567 d.f.), 
GFI = 0.667, and AGFI = 0.630. 

Discussion 

Although the Concerns Based Adoption Model has been accepted over the years 
as an appropriate framework for identifying levels of concerns for teachers, our data 
supported findings of Bailey and Palsha (1992) concerning the low reliability of the 
Awareness and Refocusing subscales. Clearly, these studies indicate that results from the 
two stages should be interpreted with extreme caution. 

Bailey and Palsha (1992) hypothesized that the Awareness subscale may not be 
needed in the model. For the first set of data from the algebra teachers, average percentile 
scores were high for both the Awareness and Information stages. The main difference 
between these two stages is that in Stage 0, Awareness, teachers may have little concern 
or involvement with the innovation, whereas in Stage 1, Information, teachers are aware 
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of the innovation and want more information. With the 35-item modified CBAM, all 
questions from the original Information stage factored to the Personal stage leaving one 
succinct entry level stage rather than two. Two questions from the original Awareness 
stage, “I'm not concerned about this innovation,” and “I am completely occupied with 
other things,” shifted to the Management Factor in the modified version. A correlation 
matrix generated from the first dataset revealed that these two questions had very low 
correlations with the other questions in the Awareness subscale (r = -.06 to r = .21). 

Also, four of the original Awareness questions are negatively stated which might cause 
confusion for teachers. Thus, the present findings indicated a lack of support for the 
Awareness subscale. 

Bailey and Palsha (1992) also hypothesized that the Refocusing subscale may not 
be needed in the model. Data from the SoCQ for the first group of algebra teachers also 
showed a high percentile level at the Refocusing stage. It would seem that teachers at a 
high Awareness/Information level would have a low percentile for Refocusing on ways to 
implement the new innovation. In the manual by Hall, George, and Rutherford (1986) 
developed for use with interpreting CBAM data, the authors discuss the interpretation of 
a “tail-up” with teachers not yet using the new innovation (non-users). Rather than 
Refocusing on the new ways to implement the innovation, non-users with high 
Awareness and Information levels, low Consequence and Collaboration levels and high 
Refocusing, may have ideas other than the new innovation to implement. In other words, 
these teachers are likely to be negative toward the innovation and not want to change. 

This difficulty in interpreting a high percentile for the Refocusing stage may help explain 
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low reliabilities for this subscale. Staff development efforts utilizing the original CBAM 
need to be aware of the low reliability results found in this study and in that of Bailey and 
Palsha. 

The reliability estimates found in this study were acceptable for the 35-item, five 
factor questionnaire produced from the exploratory factor analysis of Bailey and Palsha 
(1992). However, the first phase confirmatory factor analysis demonstrated that the 
validity of the model was less than optimal for the first group of algebra teachers. Results 
from the analysis indicated a need for modifications to the 35-item modified CBAM. 
Thus, a more parsimonious questionnaire was generated by the present authors in the 
form of a 27-item SoCQ. This model was then examined to determine whether it was 
more valid than the versions previously tested for measuring teacher concerns. 

Results from the second phase of the study showed that the 27-item SoCQ 
continued to be more reliable than Bailey and Palsha's five factor versions. However, the 
validity of the 27-item instrument was still problematic. Even though the five factor 
model seemed conceptually reasonable, it was not clear from the analysis what the factors 
represented. These results seem to indicate that the CBAM needs to be modified in 
terms of the questions being asked, not simply the grouping or number of the questions. 
The data for the second sample continued to reflect that the questions themselves appear 
to be only weakly related to the hypothesized stages of concern. With regard to the 27- 
item SoCQ, it is as if we now have a room full of weather forecasters who are in 
complete agreement that tomorrow’s weather will be warm and sunny (good reliability), 
but when tomorrow comes the weather is cold and rainy (weak validity). Needless to say, 
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given the reality of the widespread use of the SoCQ in a variety of professional 
development settings, this result is unsettling. Staff developers who use the existing 
SoCQ as a diagnostic or evaluative tool should be wary of the potential difficulties of 
interpreting project evaluation data generated by the questionnaire and the potential 
pitfalls of using the data for modifying training to better address teacher concerns. 

Despite the difficulties with the original SoCQ demonstrated by both this study 
and that of Bailey and Palsha (1992), research on staff development has shown that 
teachers do have differing stages of concern as they implement a new innovation (Friel, 
1993; Fullan, 1991; Guskey, 1986). It is tempting to look for a way to quickly and easily 
gather quantitative data, especially for large groups, that reflects the level of these 
concerns. However, this may either be much more difficult than it appears, or simply 
impossible. One viable solution to the difficulty of obtaining data on teacher concerns is 
for staff development leaders to concentrate some of their efforts on acquiring more 
qualitative information. Loucks-Horsley and Stiegelbauer (1991) recognizes that even 
valid information gathered from the SoCQ does not help to identify what the precise 
concerns are for teachers, and so they recommend the use of open-ended questions as 
well as informal discussions for pinpointing more accurately the critical issues in a 
reform effort. 

Crawford, Newberry and Fimbel (1996) reported on a project aimed at identifying 
middle school teacher concerns about an innovation through the use of teacher journals. 
The authors analyzed journal entries from 12 teachers who attended a two week 
professional development workshop on new methods for teaching pre-algebra. The 
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authors classified concerns according to five of the stages of concern from the original 
CBAM, Information, Personal, Management, Consequences and Collaboration, and also 
went beyond the CBAM to attempt to classify new categories as they emerged from the 
data. New categories included difficulty of implementation, preparedness for 
implementation, planning for implementation, and the new curriculum itself. One 
difficulty of implementation teachers identified was a lack of direct transfer for students 
between the use of manipulatives and the formation of abstract concepts (e.g., the concept 
of a variable). As the authors note, this example provides rich information about teacher 
concerns. The depth of this information would allow workshop leaders to move beyond a 
simple understanding such as “Some teachers are concerned about implementing the 
innovation,” to a more substantive consideration of the types of prescriptive methods that 
might be used to assist teachers in implementation. In general, through the use of open- 
ended questions and journal entries, staff development coordinators can be more certain 
of their diagnosis of teacher concerns and therefore more confident in their methods for 
addressing those concerns. 

Summary 

Results of this study have significant implications for staff developers and 
researchers using the SoCQ. Data generated from the SoCQ need to be interpreted with 
caution, and careful redesign of questions and subscales of the SoCQ should be 
considered in order to more accurately measure the Stages of Concern. Qualitative data 
from open-ended questions or journals might be considered as a means of gathering 
detailed information concerning issues teachers face as they implement an innovation. 
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